feat: allow overriding withCDN via env-var #55

SgtPooki · 2025-09-30T12:12:19Z

> npm run build && WITH_CDN=true npx filecoin-pin add ~/Downloads/images/IMG_1341.png


> [email protected] build
> tsc

┌  Filecoin Pin Add
│
◇  ✓ File validated (17.8 MB)
│
◇  ✓ Connected to calibration
│
◇  ✓ Payment capacity verified
│
◇  ✓ File packed with root CID: bafybeierfvvhvb73ntvschtheysvpscpynjuxgsi7jtst4wg3n3jittuca
│
◇  ✓ IPFS content loaded (17.8 MB)
│
◇  ✓ Payment capacity verified
│
◇  ✓ Storage context ready
│
◇  ━━━ Add Complete ━━━
│
│  Network: calibration
│  
│  Add Details
│    File: /Users/sgtpooki/Downloads/js-colo-media/IMG_1341.png
│    Size: 17.8 MB
│    Root CID: bafybeierfvvhvb73ntvschtheysvpscpynjuxgsi7jtst4wg3n3jittuca
│  
│  Filecoin Storage
│    Piece CID: bafkzcibfqh67ubquflgzvzb76hayv6o4yqer2wpa7susa6xna3yfktvb5zxv37nbvqbq
│    Piece ID: 0
│    Data Set ID: 244
│  
│  Storage Provider
│    Provider ID: 3
│    Name: ezpdpz-calib
│    Direct Download URL: https://calib.ezpdpz.net/piece/bafkzcibfqh67ubquflgzvzb76hayv6o4yqer2wpa7susa6xna3yfktvb5zxv37nbvqbq
│  
│  Transaction
│    Hash: 0x88314b2d3f576fb0a248f82be0971cc77a4b68ad67acc3df4db013058ff6538e
│
└  Add completed successfully

> npx filecoin-pin data-set 244

┌  Filecoin Onchain Cloud Data Sets
│
◇  ━━━ Data Sets ━━━
│
│  Data Set #244 • live
│    Managed by Warm Storage: yes
│    CDN add-on: enabled
│    Pieces stored: 1
│    Leaf count: 588804
│    Total size: 18.0 MB
│    Client data set ID: 3
│    PDP rail ID: 485
│    CDN rail ID: 487
│    Cache-miss rail ID: 486
│    Payer: 0x44f08D1beFe61255b3C3A349C392C560FA333759
│    Payee: 0xa3971A7234a3379A1813d9867B531e7EeB20ae07
│    Service provider: 0xa3971A7234a3379A1813d9867B531e7EeB20ae07
│    Provider: ezpdpz-calib (ID 3)
│    Commission: 0.00%
│  
│  Provider Service
│    Service URL: https://calib.ezpdpz.net
│    Min piece size: 1.0 KB
│    Max piece size: 32.0 GB
│    Storage price: < 0.0001 USDFC/TiB/month
│    Min proving period: 30 epochs
│    Location: unknown
│    Payment token: USDFC (native)
│  
│  Metadata
│      source: filecoin-pin
│      withCDN: (empty)
│      withIPFSIndexing: (empty)
│  
│  
│  Pieces
│    Total pieces: 1
│    Unique CommPs: 1
│    Unique root CIDs: 1
│  
│    #0
│      CommP: bafkzcibfqh67ubquflgzvzb76hayv6o4yqer2wpa7susa6xna3yfktvb5zxv37nbvqbq
│      Root CID: bafybeierfvvhvb73ntvschtheysvpscpynjuxgsi7jtst4wg3n3jittuca
│
└  Data set inspection complete

README.md

SgtPooki

self review

jennijuju · 2025-09-30T12:29:55Z

if we were going to add this. 2 TODOs needs to be considered before this is merged is:

the cost of with cdn add on is different, we will need to account for it filecoin-pin payment deposit. I won’t suggest doing this change now because filcdn is changing how they are billing (transit from a fixed fee to an egress based fee)
filecoin-pin get should fetch from filcdn link first if cdn service is purchased

SgtPooki · 2025-09-30T12:51:13Z

@jennijuju I could add a warning in here during add/import to show users that withCDN isn't fully priced throughout the codebase and link to a tracking issue. Would you like me to do that so we don't miss those follow-up actions?

bajtos · 2025-09-30T13:04:40Z

The first version looks great to me! I would like filecoin-pin to print the FilBeam retrieval URL when CDN is enabled, but that can be added later.

I did not realise there are implications for filecoin-pin payment deposit. Showing a warning to users seems reasonable enough to me. 👍🏻

SgtPooki · 2025-09-30T22:19:14Z

opened up todos: #61 and have a warning with e687b57 (#55):

> npm run build && WITH_CDN=true npx filecoin-pin add ~/Downloads/js-colo-media/71713366785__2CBE4B4C-4404-470F-AEAD-DFF56D011409.png 

> [email protected] build
> tsc

┌  Filecoin Pin Add
│
▲  CDN Pricing Notice
│
│  
│  Filecoin-pin currently does not support CDN pricing in payment calculations.
│  
│  This means:
│    • Deposit calculations may not be accurate for CDN storage
│    • You may need additional USDFC deposits for CDN-enabled uploads
│    • Filcdn is transitioning to egress-based billing (from fixed fees)
│  
│
◇  Do you want to proceed with CDN-enabled upload?
│  No
└  Add cancelled

Add failed: CDN pricing limitations warning cancelled

and for car import:

> [email protected] build
> tsc

┌  Filecoin Pin CAR Import
│
▲  CDN Pricing Notice
│
│  
│  Filecoin-pin currently does not support CDN pricing in payment calculations.
│  
│  This means:
│    • Deposit calculations may not be accurate for CDN storage
│    • You may need additional USDFC deposits for CDN-enabled uploads
│    • Filcdn is transitioning to egress-based billing (from fixed fees)
│  
│
◇  Do you want to proceed with CDN-enabled upload?
│  No
└  Import cancelled

Import failed: CDN pricing limitations warning cancelled

BigLep

I think this environment variable option for now is fine. It unblocks being able to test Filecoin Beam with filecoin-pin data.

What we really need though is a followup issue about adding "withCdn" support in general. This means exposing i through the CLI as an config option, spitting out filecoinbeam URLs, etc. plus handling the payment issues highlighted in #61.

I will get that issue created later today.

bajtos

Awesome, thank you @SgtPooki! ❤️

rvagg · 2025-10-06T07:15:43Z

Quick mention in here (without looking at this), that withCDN changes the pricing. So far we've been avoiding the complexity of different prices for storage by just avoiding withCDN entirely. Now that it's in scope we need to think about how to present the pricing info, the optionality and how to adjust to user request to "upgrade".
Ideally this would be a simple add-on, perhaps even a default so you automatically sign up for your own custom IPFS gateway, but we need to prove this out first.

rvagg · 2025-10-06T09:23:05Z

I see I'm kind of duplicating Jen's comment about pricing already .. so my drive-by probably isn't very helpful here!

SgtPooki · 2025-10-06T12:37:25Z

@rvagg so are you good with us merging this so @bajtos can use and test (likely along with provider override)?

rvagg

ok, I don't love the optionality in envToBool but whatever

rvagg · 2025-10-07T10:05:35Z

README.md

 DATABASE_PATH=./pins.db        # SQLite database
 CAR_STORAGE_PATH=./cars        # CAR file directory
 LOG_LEVEL=info                 # Logging level
+WITH_CDN=true                  # Enable/Disable CDN for content uploads (default: false)


Suggested change

WITH_CDN=true # Enable/Disable CDN for content uploads (default: false)

https://filecoinproject.slack.com/archives/C095WFA0QK1/p1759826627852839

SgtPooki added 2 commits September 30, 2025 08:11

feat: allow overriding withCDN via env-var

14ddf56

chore: fix lint

0b82e20

SgtPooki commented Sep 30, 2025

View reviewed changes

README.md Outdated Show resolved Hide resolved

SgtPooki commented Sep 30, 2025

View reviewed changes

docs: indicate default value for WITH_CDN

e37c582

fix: Warn users when enabling WITH_CDN

e687b57

BigLep approved these changes Oct 1, 2025

View reviewed changes

BigLep mentioned this pull request Oct 2, 2025

Filecoin Beam ("withCdn") support #61

Open

bajtos approved these changes Oct 3, 2025

View reviewed changes

rvagg approved these changes Oct 7, 2025

View reviewed changes

rvagg reviewed Oct 7, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: allow overriding withCDN via env-var #55

feat: allow overriding withCDN via env-var #55

SgtPooki commented Sep 30, 2025 •

edited

Loading

Uh oh!

Uh oh!

SgtPooki left a comment

Uh oh!

jennijuju commented Sep 30, 2025

Uh oh!

SgtPooki commented Sep 30, 2025

Uh oh!

bajtos commented Sep 30, 2025

Uh oh!

SgtPooki commented Sep 30, 2025

Uh oh!

BigLep left a comment

Uh oh!

bajtos left a comment

Uh oh!

rvagg commented Oct 6, 2025

Uh oh!

rvagg commented Oct 6, 2025

Uh oh!

SgtPooki commented Oct 6, 2025

Uh oh!

rvagg left a comment

Uh oh!

rvagg Oct 7, 2025

Uh oh!

Uh oh!

feat: allow overriding withCDN via env-var #55

Are you sure you want to change the base?

feat: allow overriding withCDN via env-var #55

Conversation

SgtPooki commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

SgtPooki left a comment

Choose a reason for hiding this comment

Uh oh!

jennijuju commented Sep 30, 2025

Uh oh!

SgtPooki commented Sep 30, 2025

Uh oh!

bajtos commented Sep 30, 2025

Uh oh!

SgtPooki commented Sep 30, 2025

Uh oh!

BigLep left a comment

Choose a reason for hiding this comment

Uh oh!

bajtos left a comment

Choose a reason for hiding this comment

Uh oh!

rvagg commented Oct 6, 2025

Uh oh!

rvagg commented Oct 6, 2025

Uh oh!

SgtPooki commented Oct 6, 2025

Uh oh!

rvagg left a comment

Choose a reason for hiding this comment

Uh oh!

rvagg Oct 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SgtPooki commented Sep 30, 2025 •

edited

Loading